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INTRODUCTION 


This  document  describes  ReqData,  a  package  for  requesting  and  receiving  seismic 
waveform  data  from  Internet  sites  running  an  autoDRM  1  (automatic  Data  Request 
Manager;  Kradolfer,  1993).  The  ReqData  package  simplifies  the  task  of  formatting 
the  email  requests  for  the  user’s  list  of  events  and  stations.  ReqData  automatically 
parses  the  email  responses  from  the  autoDRM,  converts  GSE2.0  format  to  CSS3.0 
and  installs  data  in  directories  as  specified  by  the  user.  Also  included  with  ReqData 
is  a  GUI  for  managing  the  progress  of  the  responses.  To  help  manage  station  files, 
two  programs  are  included  in  the  ReqData  package:  one  for  querying  the  autoDRM 
about  stations  and  one  to  parse  the  responses  into  CSS3.0  station  files. 

Parameters  to  the  waveform  request  program,  reqdata  are:  an  approximate  time 
and  location  of  events  to  request,  a  list  of  stations  and  channels  to  request  and  the 
directory  in  which  to  install  the  data.  To  limit  the  size  of  the  individual  email  re¬ 
sponses,  requests  are  formed  separately  for  each  station.  A  status  file  and  a  log  file  are 
created  in  the  data  directory  using  the  data  prefix  specified  by  the  user.  The  log  file 
initially  contains  a  copy  of  the  autoDRM-formatted  email  requests  sent  to  the  speci¬ 
fied  autoDRM  site.  The  status  file  records  the  requested  times  for  each  channel,  the 
date  of  the  request  and  the  status  (waiting,  received-data,  no-data-available, 
etc.).  As  email  responses  are  received  from  the  autoDRM,  both  the  log  and  status 
files  are  updated. 

The  log  and  status  files  are  ASCII  text  and  can  be  viewed  by  hand.  An  X- 
Windows  program,  reqstat,  is  provided  as  a  convenient  tool  for  viewing  these  files. 
With  reqstat,  the  user  can  easily  see  the  status  of  all  event-requests  and  the  status 
of  all  individual  channel-requests.  From  reqstat,  requests  can  be  resubmitted  to  the 
same  or  a  different  autoDRM  site. 

The  software  that  automatically  parses  GSE2.0-formatted  responses  from  an  au¬ 
toDRM  consists  of  three  parts:  a  mail  delivery  program  named  deliver2,  the  shell 
script  that  controls  deliver,  named  .deliver,  and  the  conversion  programs  gse2css 
and  gse2site. 

After  ReqData  is  installed,  a  user’s  incoming  email  is  automatically  filtered  by 
the  .deliver  script  to  catch  all  GSE2.0  formatted  messages.  These  messages  are 
removed  from  the  user’s  email  and  processed  by  ReqData  conversion  programs.  All 

1Since  no  strict  standards  exist  for  autoDRM,  ReqData  may  not  work  equally  well  at  all  sites. 
ReqData  was  designed  and  tested  primarily  with  the  autoDRM  implemented  at  USGS  and  uses 
GSE2.0  autoDRM  commands  defined  in  CRP  243. 

2Chip  Salzenberg,  ComDev/TC  Telemanagement 
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messages  not  recognized  as  GSE2.0  are  forwarded  to  the  user’s  regular  mailbox. 

At  the  time  of  request,  each  request  is  logged  and  given  a  unique  request-id  that 
is  returned  by  the  autoDRM  as  the  REF.ID  in  the  email  response.  The  REF-ID  allows 
the  email  parsing  programs  to  match  incoming  data  with  the  request-ids  and  install 
the  data  in  the  appropriate  directory. 

The  .  deliver  script  compares  the  REF_ID  of  each  GSE2.0  message  to  the  request 
log  created  by  reqdata.  If  the  REF_ID  is  valid,  a  conversion  program  is  executed  to 
process  the  message.  If  the  REF-ID  of  a  GSE2.0  message  is  not  found  in  the  request 
log,  the  message  is  forwarded  to  the  user’s  regular  mailbox.  It  is  an  installation 
option  to  have  all  processed  GSE2.0  email  messages  either  saved  in  a  file  or  discarded. 
When  any  message  fails  to  be  processed  for  any  reason,  all  processing  error  messages 
are  logged  and  the  message  can  be  either  forwarded  to  the  user’s  mailbox  or  saved, 
depending  on  installation  options. 

Each  site  operating  an  autoDRM  places  restrictions  on  the  size  of  the  email  re¬ 
sponse  to  the  user.  In  order  to  facilitate  email  transfer  of  large  responses,  some  sites 
employ  the  CONTINUE  command,  which  is  part  of  the  GSE2.0  autoDRM  command 
set.  The  .deliver  script  can  handle  autoDRM  responses  that  are  continued  over  two 
or  more  separate  email  messages.  Frequently,  the  individual  messages  of  a  contin¬ 
ued  response  will  arrive  out  of  order.  The  .deliver  script  holds  continued  messages 
until  the  first  message  of  the  response,  which  contains  the  REF_ID,  is  received  before 
processing  all  the  continuations. 

The  waveform  parsing  program,  gse2css,  converts  GSE2.0  WID2  headers  and 
CM6  compressed  waveform  data  to  CSS3.0  format.  An  origin  file  is  created  from 
the  event  information  input  at  request  time.  Individual  waveform  files  are  created 
for  each  channel  following  the  naming  convention  of  sta.chan.epochaltime.w.  If  a 
request  is  repeated,  the  incoming  data  will  write  over  the  older  data  if  the  sta,  chan 
and  epochal  time  are  the  same. 

Another  request  program,  reqchan,  requests  station  and  channel  information 
from  an  autoDRM.  When  the  .deliver  script  finds  keywords  DATA-TYPE  STATION 
or  DATA-TYPE  CHANNEL  in  the  autoDRM  response,  a  different  conversion  program, 
named  gse2site,  is  executed.  Gse2site  creates  or  updates  the  system’s  CSS3.0 
site  and  sitechan  tables  if  new  station  and/or  channel  information  arrives  from  an 
autoDRM. 

ReqData  supports  multiple,  simultaneous  users.  A  CSS3.0  lastid  table-file  is  up¬ 
dated  with  orid,  wfid,  chanid  and  msgid  records.  A  central  request-file  logs  request-ids 
for  all  users.  A  Unix  file  locking  mechanism  insures  that  all  files  being  written  to  by 
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the  message  conversion  programs,  including  lastid,  origin,  wfdisc,  status  and  data 
files,  are  updated  correctly.  This  is  necessary  not  only  to  allow  multiple  users,  but 
even  for  a  single  user,  since  the  processing  of  sequentially  arriving  email  messages 
can  overlap  in  time,  resulting  in  the  execution  of  multiple  instances  of  the  conversion 
programs. 


INSTALLATION 
Getting  The  Package 

The  ReqData  package  can  be  obtained  by  anonymous  FTP  from  es2.multimax.com. 
In  the  directory  pub/gtdb/reqdata,  you  will  find  compressed  tar  files  containing 
executables  for  SunOS-4.1.3,  Sun-5.3  (Solaris-2.3),  and  IRIX-5.3: 

reqdat al . 0 . bin . sun4 . 1 . 3 . Z 
reqdat al . 0 . bin . sun5 . 3 . Z 
reqdat al .0. bin. irix5. 3. Z 

These  tar  files  also  contain  tables  of  station  locations,  travel  times  and  autoDRM 
addresses.  This  document  is  included  as  a  PostScript  file,  and  there  are  Unix  “man” 
pages  for  the  programs.  The  complete  source  code  is  also  available  as  a  separate 
compressed  tar  file: 

reqdatal . 0 . src .  Z 

The  directories  in  the  ReqData  package  (excluding  the  source)  are: 

reqdata/bin 
reqdat  a/ c  ont inued 
reqdata/doc 
reqdata/info 
reqdata/logs 
reqdat a/man/manl 


3 


reqdata/man/man3 
reqdata/man/man5 
reqdata/save 
reqdata/tables/ static 
reqdata/tables/dynamic 

Install  this  directory  tree  in  a  location  available  to  all  users  and  make  sure  that 
the  directories  tables/dynamic,  logs,  continued,  and  save  are  writable  by  all 
users  of  the  package.  Add  reqdata/bin  to  each  user’s  PATH  environment  variable 
and  add  reqdata/man  to  the  MANPATH  environment  variable.  Each  user  must  also  set 
the  environment  variable  REQDATA_HOME  to  the  location  of  the  reqdata  directory. 


Activating  Automatic  Email  Parsing 

There  are  just  a  few  steps  necessary  to  activate  the  automatic  parsing  of  GSE- 
formatted  email.  First,  edit  the  file  reqdata/. deliver  and  change  the  line 
REQDATA_HOME=/path/reqdata  to  be  the  actual  location  of  the  reqdata  directory. 
Then  copy  reqdata/. deliver  to  each  user’s  home  directory  (or  make  a  link  to  it). 
Create  a  file  named  .forward  in  each  user’s  home  directory  containing  the  line: 

"  |  /paifi/reqdata/bin/deliver  username  II  exit  75". 

Include  the  quotes.  Substitute  for  /path/ reqdata  the  actual  location  of  reqdata  and 
substitute  for  username  the  user’s  login  name.  Mail  must  be  deliverable  locally  to 
username,  so  it  cannot  be  an  address  to  another  machine  (No  @).  If  the  user’s  mailbox 
file  is  not  in  the  directory  /var/spool/mail,  then  specify  the  mailbox  directory  with 
a  -m  option.  For  example,  if  the  user’s  mailbox  file  is  in  the  directory  /var/mail, 
then  create  a  .forward  file  with: 

"  I /pat/i/reqdata/bin/deliver  username  -m  /var/mail  II  exit  75". 


Leave  a  space  between  the  -m  and  /var/mail. 

The  .forward  file  will  instruct  the  Unix  mail  receiving  program  (sendmail)  to 
execute  the  deliver  program  and  pass  all  email  to  it.  The  deliver  program  will 
use  the  .deliver  script  to  detect  GSE  formatted  mail  and  execute  the  appropriate 
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parsing  program.  If  reqdata/bin/deliver  is  unavailable  (perhaps  because  a  file 
server  is  down  or  an  automounter  failed),  the  “I  I  exit  75”  in  the  .forward  file 
instructs  sendmail  to  requeue  mail  for  later  delivery.  Normally,  this  is  a  sufficient 
safeguard  against  bouncing  mail  back  to  an  autoDRM.  But  if  the  deliver  program 
is  unavailable  for  a  long  time  (more  than  a  few  hours),  sendmail  will  reply  to  senders 
that  it  is  having  a  delivery  problem  and  eventually  return  messages  to  senders. 


Optional  Installation  Step 

The  following  optional  installation  step  will  prevent  mail  from  being  returned  to  the 
sender  when  the  deliver  program  is  unavailable  for  a  long  time.  Instead  of  requeueing 
messages,  we  can  instruct  the  sendmail  program  to  bypass  the  .  forward  file  and  send 
mail  directly  to  the  user’s  mailbox  whenever  the  deliver  program  is  unavailable. 
First  remove  the  “I  I  exit  75”  from  the  .forward  file.  Edit  the  sendmail  aliases 
file,  usually  /etc/aliases.  (You  will  probably  need  root  permission  to  edit  this  file. 
If  it  is  not  in  /etc,  check  for  its  location  in  the  configuration  file  /etc/sendmail .  cf .) 
Look  for  a  line  in  /etc/aliases  that  begins  with  username :,  where  username  is  your 
login  name.  After  that  line,  add  (substituting  your  login  name  for  username ): 

owner -username :  \username 

If  there  isn’t  a  line  beginning  with  username :,  then  add  the  following  two  lines  to 
the  file: 

username :  username 
owner-username :  \username 

The  “ owner-username :  ”  line  in  the  aliases  file  will  prevent  email  from  being  bounced 
back  to  the  sender,  in  the  event  that  the  deliver  program  is  unavailable  or  fails  to 
execute  correctly.  If  deliver  fails  for  any  reason,  sendmail  will  send  the  mail  along 
with  an  error  message  directly  to  the  user’s  mailbox. 


Testing  The  Installation 

It  is  a  good  idea  to  test  the  installation  “locally”  before  sending  a  request  to  an 
autoDRM.  After  you  have  completed  the  installation  steps  of  the  previous  section 
(installed  the  reqdata  directory,  edited  and  installed  the  .deliver  and  .forward 
files),  try  the  following  test: 
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setenv  REQDATA.HOME  /path/ reqdata 
cd  /path/ reqdata 
bin/reqdata  par=test_parf ile 

The  file  test_parf  ile  contains: 

stachanlist=AAE/B* 

address=user 

r et  addr =me@mymach ine . address 
basedir=/ tmp/ t est_reqdat a 
time=96/06/19  00:18:02 
lat=36. 11 
lon=35 . 80 
dept h= 10 . 0 

This  will  instruct  reqdata  to  form  a  request  and  mail  it  to  you.  Check  your  mail 
for  the  test  request  message.  If  you  do  not  receive  it  after  a  few  moments,  look  in 
logs/deliver. log  for  error  messages  and  check  the  mail  queue  to  see  if  the  test 
message  could  not  be  delivered.  If  you  do  receive  the  test  request  message,  you  can 
delete  it  and  continue.  Check  to  see  if  the  directory  /tmp/test_reqdata  was  created, 
and  if  it  is  there,  proceed  with: 

bin/reqstat  /tmp/test_reqdata  & 

The  reqstat  main  window  should  display  one  event  in  its  Requested  Events  list 
with  status  “waiting”  and  prefix  “test  .reqdata” .  Select  the  event  with  a  mouse  click 
and  select  the  Stations...  button  to  display  the  channel  listing.  There  should  be 
three  lines  for  AAE  and  channels  BHE,  BHN  and  BHZ. 

Continue  the  test  of  the  mail  parsing  programs  by  mailing  a  GSE-formatted  mes¬ 
sage  to  yourself: 

mail  username  <  testunsg 
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After  a  few  seconds,  the  reqstat  status  fields  for  channels  BHN  and  BHZ  should  change 
to  “response” ,  and  the  files 

/tmp/test_reqdata/w/AAE .  BHE .  835143783 . 02 .  w 
/tmp/test_reqdata/ w/AAE .  BHZ .  835143783 . 02 .  w 

should  be  created.  If  they  are  there,  the  installation  is  good. 


SENDING  REQUESTS  WITH  ReqData 
Basic  Operation 

The  program  reqdata  requests  waveform  data  from  one  or  more  autoDRM’s,  given 
an  approximate  event  time  and  location  and  a  list  of  stations.  This  information  can 
be  input  on  the  command  line,  or  the  user  can  place  arguments  in  a  file  and  input 
the  filename  on  the  command  line  with  reqdata  par=parfile.  A  simple  parf  ile 
for  reqdata  looks  like: 

r e t  addr=us  ername  Omyaddr e s  s 
basedir=/diskl/data/ eventOl 

stachanList=AAM/ * , ALQ/B* , BLA/BHZ , NORES/ * , ESDC/b* 

time=96/05/13  04:53:47 

lat=7 . 19N 

lon=76.88W 

depth=27 . 0 

Specify  your  email  address  with  the  retaddr  argument,  and  specify  the  directory 
where  the  data  will  be  installed  with  the  basedir  argument.  The  stachanList  argu¬ 
ment  is  a  list  of  station/channel  or  network/element  pairs.  The  list  can  contain  the 
wildcard  character  as  the  final  character  of  the  channel  or  array  element  name. 
When  a  wildcard  character  is  encountered,  the  program  first  searches  the  affiliation 
table  $REQDATA_HOME/tables/static/global  .affiliation  for  a  matching  network 
name.  In  the  example  above,  NORES/*  will  expand  to  include  all  the  elements  of 
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the  NORES  array  and  all  the  channels  of  each  element.  Array  expansion  can  also  be 
limited.  In  the  example  above,  ESDC/b*  expands  to  include  only  the  broad-band  chan¬ 
nels  at  the  ESDC  array:  ESLA/BHZ,  ESLA/BHN,  ESLA/BHE.  The  program  also  searches 
the  sitechan  file  $REQDATA_HOME/tables/static/global.sitechan  for  a  matching 
station  name,  and  if  found  expands  the  character  to  include  all  channels  for  the 
matching  station.  A  AM/*  expands  to  include  all  channels  AAM/HLZ,  AAM/HLN,  AAM/HLE, 
AAM/BLZ,  AAM/BLN,  AAM/BLE,  AAM/LLZ,  AAM/LLN,  AAM/LLE,  whereas  ALQ/B*  expands 
to  only  the  broadband  channels  ALQ/BHZ,  ALQ/BHN,  ALQ/BHE.  Station  and  channels 
names  are  case  insensitive,  so  for  example,  you  could  specify  alq/b*  instead  of  ALQ/B*. 

Input  an  approximate  event  time  and  location  with  the  arguments  time,  lat, 
Ion  and  depth.  The  format  for  time  is  yyyy/mm/dd  hh:mm:ss.s,  but  reqdata  also 
recognizes  a  two  digit  year-1900,  as  shown  above,  or  an  epochal  time  as  in  CSS3.0 
origin  tables.  Input  the  latitude  and  longitude  using  N,  S,  E,  W  for  north,  south,  east 
or  west  or  input  a  positive  number  for  north  and  east  and  negative  number  for  south 
and  west. 

Reqdata  searches  $REQDATA_HOME/tables/static/global.site  for  station  loca¬ 
tions  and  computes  request  time  windows.  The  default  time  window  for  each  station 
starts  one  minute  before  the  first  P  arrival  time  and  ends  30  minutes  after  the  LR  ar¬ 
rival.  The  user  can  change  the  default  by  using  the  optional  arguments  begPhase  and 
endPhase.  The  format  for  these  arguments  is  phase+/-seconds.  Using  this  format, 
the  default  time  window  looks  like: 

begPhase=P-60 . 
endPhase=LR+1800 . 

To  request  a  four-minute  time  window  centered  on  the  P  arrival  time,  for  example, 
add  the  following  lines  to  the  paxf  ile: 

begPhase=P-120 . 
endPhase=P+120 . 

Other  standard  phases  in  the  IASPEI  table  are  also  recognized  for  begPhase  and 
endPhase. 

Once  reqdata  has  computed  time  windows  for  all  station/ channel  pairs,  it  searches 
the  address  file  $REQDATA_HOME/tables/static/global. address  for  the  address  of 
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an  autoDRM  associated  with  each  individual  channel.  The  address  file  contains  free- 
formatted  lines  with  sta,  chan,  address,  format  information.  For  example,  the 
following  lines 

ALQ  BHZ  autodrm@gldfs.cr.usgs.gov  GSE2.0 
ALQ  BHN  autodrm@gldfs.cr.usgs.gov  GSE2.0 
ALQ  BHE  autodrm@gldfs.cr.usgs.gov  GSE2.0 
NRAO  she  messages@cdidc.org  GSE2.0 
NRAO  shn  messages@cdidc . org  GSE2 . 0 
NRAO  shz  messages@cdidc.org  GSE2.0 
NRA1  shz  messages@cdidc.org  GSE2.0 
NRA1  shz  messages@cdidc.org  GSE2.0 


instruct  reqdata  to  request  ALQ  broadband  channels  from  autodrm@gldf  s .  cr .  usgs .  gov 
and  request  NORES  channels  from  messages@cdidc .  org.  If  two  different  addresses  are 
listed  in  the  address  file  for  one  station/channel,  reqdata  uses  the  first  line  encoun¬ 
tered. 


Optional  Arguments  to  ReqData 

stachanFile  This  optional  argument  can  be  used  in  place  of 
stachanList  to  specify  a  list  of  stations  and  channels  to  request.  Simply  list 
the  station/channel  and  network/element  pairs  in  a  file,  one  pair  per  line,  and 
input  the  filename  using  stachanFile=filename.  Only  the  first  two-columns 
of  this  file  are  used  by  reqdata.  Everything  on  the  line  after  the  channel  name 
is  ignored.  The  wildcard  character  can  be  used  in  the  file.  The  '#’  character 
can  be  used  to  comment-out  lines.  This  allows  one  to  easily  use  a  channel  list¬ 
ing  obtained  from  an  autoDRM  (see  reqchan  below)  as  a  stachanFile,  which 
might  look  like: 


#autodrm@gldf s . cr . usgs . gov 


#Sta 

Chan  Aux 

Latitude 

Longitude 

Elev 

Depth 

Hang 

Vang 

AAE 

BHZ 

IU 

9.02917 

38.76556 

2.442 

0.000 

0.0 

0.0 

AAE 

BHN 

IU 

9.02917 

38.76556 

2.442 

0.000 

0.0 

90.0 

AAE 

BHE 

IU 

9.02917 

38.76556 

2.442 

0.000 

90.0 

90.0 

AAE 

LHZ 

IU 

9.02917 

38.76556 

2.442 

0.000 

0.0 

0.0 

AAE 

LHN 

IU 

9.02917 

38.76556 

2.442 

0.000 

0.0 

90.0 

AAE 

LHE 

IU 

9.02917 

38.76556 

2.442 

0.000 

90.0 

90.0 
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AAM 

HLZ 

US 

42.29972 

-83.65611 

0.249 

0.000 

0.0 

0.0 

AAM 

HLN 

US 

42.29972 

-83.65611 

0.249 

0.000 

0.0 

90.0 

AAM 

HLE 

US 

42.29972 

-83.65611 

0.249 

0.000 

90.0 

90.0 

AAM 

BLZ 

US 

42.29972 

-83.65611 

0.249 

0.000 

0.0 

0.0 

AAM 

BLN 

US 

42.29972 

-83.65611 

0.249 

0.000 

0.0 

90.0 

AAM 

BLE 

US 

42.29972 

-83.65611 

0.249 

0.000 

90.0 

90.0 

prefix  When  reqdata  is  executed  it  creates  the  files  prefix. reqlog  and  pre¬ 
fix  .  reqstatus  in  the  directory  specified  by  basedir.  When  gse2css  processes 
a  response  to  the  data  request,  it  creates  prefix. origin  and  prefix.wfdisc 
in  the  same  directory.  The  default  value  of  prefix  is  the  directory  name.  For 
example,  the  parf  ile  above  would  cause  the  following  files  to  be  created. 

/ diskl/data/ eventOl/ eventOl . reqlog 
/diskl/data/eventOl/eventOl .reqstatus 
/diskl/data/eventOl/eventOl .origin 
/diskl/data/ eventOl/ eventOl . wf disc 

dir  This  is  the  directory  where  waveform  files  are  installed  using  the  naming  con¬ 
vention  sta.chan.epochal_time.w.  It  is  relative  to  basedir,  if  it  does  not 
begin  with  7’.  The  default  for  dir  is  w.  Before  it  sends  any  requests,  reqdata 
confirms  that  the  data  directory  exists  or  can  be  created. 

mb  This  body  wave  magnitude  is  recorded  in  the  prefix. origin  file. 

ms  This  surface  wave  magnitude  is  recorded  in  the  prefix. origin  file. 

ml  This  local  magnitude  is  recorded  in  the  prefix. origin  file. 

origin  An  alternative  method  of  specifying  the  event  time  and  location  is  to  input  a 
free-formatted  CSS3.0  origin  record  with  the  origin  argument.  All  the  fields  of 
the  origin  record  must  be  specified  (null  values  may  be  substituted),  and  they 
will  be  recorded  in  the  prefix. origin  file.  The  origin  argument  can  also  be 
set  to  the  name  of  a  file  containing  one  or  more  CSS3.0  origin  records.  Data 
requests  will  be  made  for  each  origin  in  the  file. 

maxChan  The  maximum  number  of  channels  per  email  request.  The  default  is  3 
channels.  Three  email  requests  will  be  sent  for  a  station  that  has  9  channels. 

address  If  this  argument  is  specified,  all  requests  will  be  sent  to  the  value  of  address, 
and  $REQDATA_HOME/tables/ static/ global .  address  will  not  be  used. 
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ttonly  If  this  argument  is  set  to  1,  the  start  and  end  times  of  each  waveform  that 
would  be  requested  are  printed,  but  no  requests  are  sent. 

verbose  Controls  information  printed  during  execution.  Set  to  0,  1  or  2. 

start-time  Overrides  begPhase  for  the  computation  of  the  requested  waveform  start 
times.  If  specified,  this  will  be  the  start  time  of  all  waveforms  requested  for  all 
stations. 

end_time  Overrides  endPhase  for  the  computation  of  the  requested  waveform  end 
times.  If  specified,  this  will  be  the  end  time  of  all  waveforms  requested  for  all 
stations. 

sendmail  The  location  of  the  sendmail  program  can  be  specified  with  this  argument. 
The  default  value  is  /usr/lib/sendmail.  (Sendmail  is  a  standard  Unix  mail 
daemon,  responsible  for  routing  mail.) 

sleep  Time  in  seconds  that  reqdata  sleeps  between  executions  of  sendmail.  De¬ 
faults  to  two.  A  sleep  is  frequently  necessary  to  prevent  mail  queues  from  filling 
up  or  prevent  exhausting  other  Unix  system  resources,  such  as  total  number  of 
processes. 

tableDir  This  is  the  directory  containing  the  two  subdirectories 
static  and  dynamic  with  table-files  used  by  reqdata  and  associated  programs. 
If  tableDir  is  not  specified,  the  directory  $REQDATA_HOME/tables  will  be  used. 
The  following  files  are  needed  by  reqdata: 

static /global . address 
static/global . affiliation 
static/global . site 
static/global . sitechan 
static/iasp91 .hed 
static/iasp91 . tbl 

and  the  following  files  will  be  created  by  reqdata  or  conversion  programs: 

dynamic/ global . last id 
dynamic/global .request 
dynamic/global . continued 

addressTable  Overrides  the  default  address-table  file. 
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aff  iliationTable  Overrides  the  default  affiliation-table  file. 
siteTable  Overrides  the  default  site-table  file. 
sitechanTable  Overrides  the  default  sitechan-table  file. 
iaspeiTable  Overrides  the  default  IASPEI  table  prefix. 
lastidTable  Overrides  the  default  lastid-table  file. 
requestTable  Overrides  the  default  request-table  file. 

Updating  Station  Information  with  reqchan 

The  ReqData  package  comes  with  global .  site,  global .  sitechan,  global .  address 
and  global. affiliation  files  which  contain  station  and  channel  information  for 
several  autoDRM  sites.  If  the  user  wishes  to  request  data  from  stations  at  a  new 
autoDRM  site,  the  program  reqchan  can  be  used  to  update  all  of  these  station  files, 
except  global. affiliation.  Another  use  for  the  reqchan  program  is  to  update  the 
station  files  for  any  autoDRM  site  to  catch  new  stations  that  may  have  been  added 
to  the  site’s  database.  A  simple  parf  ile  for  reqchan  looks  like: 

retaddr=username@myaddress 

addr  e  s  s = aut  odrm_name  @  aut  odrm_addr  e  s  s 

log=logfile 

Reqchan  sends  a  request  for  a  complete  station  and  channel  listing  to  the  specified 
autoDRM.  The  response  is  processed  by  the  program  gse2site,  which  creates  or 
updates  the  site,  s it e chan  and  address  files.  The  response  is  also  forwarded  to  the 
user’s  mailbox,  so  it  can  be  saved  for  use  as  stachanFile  input  to  reqdata. 

MANAGING  REQUESTS  WITH  REQSTAT 
Basic  Operation 

When  data  requests  are  made  by  reqdata,  the  files  prefix. reqlog  and  prefix. - 
reqstatus  are  created  in  the  basedir  directory.  For  the  example  reqdata  parf  ile 
shown  above,  the  following  two  files  would  be  created: 
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/ diskl/dat a/ eventOl/eventOl . reqlog 
/diskl/data/eventOl/eventOl.reqstatus 

The  information  in  these  files  can  be  reviewed  with  the  program  reqstat.  You  can 
execute  reqstat  with  no  command  line  arguments,  with  a’  .reqstatus  file,  or  with 
a  directory  name.  For  example,  the  following  are  valid  execution  statements: 

reqstat 

reqstat  /diskl/data/eventOl/eventOl .reqstatus 
reqstat  /diskl/data 

If  no  files  or  directories  are  on  the  command  line,  the  user  has  the  option  to  input 
.reqstatus  files  with  the  File/Open  option,  discussed  below.  If  reqstat  is  exe¬ 
cuted  with  a  filename  as  a  command  line  argument,  it  reads  the  contents  of  that 
file  only.  On  the  other  hand,  if  a  directory  name  is  input  on  the  command  line, 
reqstat  recursively  searches  for  all  .  reqstatus  files  in  the  input  directory  and  in  all 
subdirectories  beneath  the  input  directory. 

Figure  1  shows  the  main  window  of  reqstat  displaying  a  list  of  requested  events. 
In  addition  to  the  status  of  each  event  request,  other  information  about  the  event  is 
displayed  including  time,  location,  date  requested,  and  all  fields  in  the  origin  table- 
file.  The  status  field  displays  “done”  when  autoDRM  responses  have  been  received 
for  each  individual  station  request  associated  with  the  event. 

The  status  of  individual  stations  will  be  displayed  in  another  window  when  the 
user  selects  an  event  line  and  selects  the  Stations  option  button.  Figure  2  shows  the 
reqstat  station  request  window  for  one  event.  Each  station/channel  requested  for 
the  event  is  listed.  The  status  field  displays  “waiting”  or  “response” .  The  limits 
for  the  time  window  requested  and  the  time  window  actually  received  are  displayed. 
If  no  data  has  been  received  for  a  channel,  the  received-tbeg  will  display  additional 
status  information,  such  as  NO  RESPONSE,  NOT  AVAILABLE  or  REQUEST 
ERROR.  The  address  of  the  autoDRM  to  which  the  request  was  sent  is  displayed 
along  with  the  time  of  the  last  activity. 

The  reqstat  station  request  window  has  a  Re-request  option  that  allows  the 
user  to  re-request  all  selected  channels,  all  channels  with  a  NO  RESPONSE  status, 
or  all  channels  with  a  NOT  AVAILABLE  status.  The  Re-request  option  generates 
a  Confirm  Re-request  window  that  lists  all  the  channels  that  will  be  re-requested 
and  allows  the  user  to  edit  the  time  window  limits  and  the  autoDRM  address.  Use 
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Figure  1:  The  main  window  of  reqstat.  All  requested  events  are  listed. 


Figure  2:  The  reqstat  station  request  window.  The  status  of  each  station/ channel 
request  for  one  event  is  displayed. 
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a  right-mouse-button  click  in  any  field  of  the  Confirm  Re-request  window  to  enter 
edit  mode.  An  autoDRM  address  can  be  entered  for  each  channel  or  one  can  be 
selected  from  a  list.  After  a  re-request  is  confirmed,  a  new  line  for  each  channel 
re-requested  is  be  added  to  the  station  request  window. 

From  the  reqstat  station  request  window  the  Messages  option  can  be  used  to 
display  a  copy  of  the  exact  autoDRM-formatted  request  that  was  sent  and  detailed 
information  on  the  response  that  was  received  for  the  selected  channel.  Figure  3 
shows  an  example  of  the  content  in  a  reqstat  messages  window.  The  time  that 
gse2css  was  executed  to  process  the  autoDRM  response  is  displayed  followed  by 
message  identification  lines  and  all  DATA.TYPE  LOG  lines  that  were  in  the  message. 


Reqstat  Options 

All  options  for  each  reqstat  window  are  listed  below  with  a  brief  explanation  of  their 
function. 


Main  Window 

File/Open  Displays  the  standard  Xll  file  selection  popup.  Select  an  individual 
.reqstatus  file  to  view,  or  select  a  directory  (double  click  on  the  directory 
name  in  the  Directories  list). 

File/ Warnings  Displays  a  list  of  all  warnings  encountered. 

View/ Attributes  This  popup  allows  the  user  to  customize  the  information  dis¬ 
played  for  each  requested  event.  Click  on  an  attribute  Name  to  add  or  remove 
that  attribute  from  the  display  line.  Attributes  can  be  reordered  by  deselecting 
all  of  them  and  then  selecting  in  the  order  you  want  them  displayed.  Click  in 
the  Format  field  to  edit  the  format  used  to  display  the  attribute. 

View/Clear  Remove  the  current  event  listing  from  reqstat. 

View/Sort  Select  the  sort  option  for  the  events:  sort  by  Prefix,  by  Event  Time 
or  by  Request  Date. 

View/Stations  Display  the  reqstat  station  request  window  for  the  selected  event. 

Option/Re-request  This  option  will  re-request  stations  for  the  selected  events. 
There  are  two  types  of  re-requests:  re-request  all  stations  for  which  no  re¬ 
sponse  has  been  received,  or  re-request  all  stations  for  which  a  NO  DATA 
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msgJd=Z5Z1  requested  Tue  Apr  23  09:06:09 1996 
Aisr/lib/sendmail  autodrm@gldfs.cr.usgs.gov 
BEGIN  GSEZ.0 
MSGJTYPE  REQUEST 
MSGJD  Z5Z1  reqdata 
E-MAIL  ihenson@multimax.com 
STA_LIST  ARU 
CHAN_LIST  bhz,bhn,bhe 
TIME  1996/1/28  8:47:25  TO  1 996/1  /28  9:1Z:33 
WAVEFORM  GSEZ.0 
STOP 

msgJd=Z5Z1  executing  gseZcss  at  Tue  Apr  Z3  09:16:38  1996 
MSGJYPE  DATA 

MSGJD  961 1 4  1 25748_multima  USA_NDC 
REFJD25Z1  reqdata 
DATAJTYPE  LOG 

This  message  was  generated  by  the  U.  S.  Geological  Survey  (USA_NDC) 
Automatic  Data  Request  Manager  (AutoDRM). 

Request  received  at:  Apr  23  0657:48  1996  (local). 

Processing  started  at:  Apr  23  07:07:21  1996  (local). 

907  waveform  channels  are  currently  available. 

Command  syntax  GSEZ.0  selected. 

Time  range:  Z8  JAN  1996  08:47:25.00  to  28  JAN  1996  09:12:33.00  (WAVEFORM). 

Chanhels  selected  (WAVEFORM): 

ARU:  BHZ,  BHN,  BHE 


ARU/BHZ 

ARU/BHZ 

ARU/BHN 

ARU/BKN 

ARU/BHE 

ARU/BHE 


1996/01/28  08:47:29.434 
1996/01/28  09:12:20534 
1996/01/28  08:47:29.434 
1996/01/28  09:12:20.634 
1996/01/28  08:47:29.434 
1996/01/28  09:12:20.634 


1996/01/28  0857:34.434 
1996/01/28  09:12:32.984 
1996/01/28  08:57:48.834 
1996/01/28  09:12:32584 
1996/01/28  0856:25.234 
1996/01/28  09:12:32.984 


Hide 


Figure  3:  The  reqstat  messages  window.  A  copy  of  the  autoDRM-formatted  request 
is  followed  by  information  resulting  from  the  processing  of  the  autoDRM  response. 
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AVAILABLE  response  was  received.  Before  the  re-requests  are  actually  sent, 
a  Confirm  Re-request  window  will  list  the  channels  that  will  be  re-requested. 
The  requested  time  limits  and  the  autoDRM  addresses  can  be  changed  before 
a  Confirm  option  sends  the  re-requests.  A  Cancel  button  is  also  available. 


Station  Request  Window 

View/ Attributes  This  popup  allows  the  user  to  customize  the  information  dis¬ 
played  for  each  requested  station/channel.  Click  on  an  attribute  Name  to  add 
or  remove  that  attribute  from  the  display  line.  The  attributes  can  be  reordered 
by  deselecting  all  of  them  and  then  selecting  each  attribute  in  the  order  you 
want  them  displayed. 

View/Messages  Display  the  exact  GSE-formatted  request  message  that  was  sent 
to  the  autoDRM  and  display  any  DATA-TYPE  LOG  or  DATA_TYPE 
ERROR  messages  received  from  the  autoDRM.  Error  messages  from  gse2css 
are  also  displayed. 

Option/Re-request  This  option  will  re-request  data  for  the  specified  stations  or 
channels.  There  are  three  types  of  re-requests:  re-request  all  selected  stations, 
re-request  all  stations  for  which  no  response  has  been  received,  or  re-request  all 
stations  for  which  a  NO  DATA  AVAILABLE  response  was  received.  Before 
the  re-requests  are  actually  sent,  a  Confirm  Re-request  window  will  display 
all  the  stations  and  channels  that  will  be  re-requested  along  with  Confirm  and 
Cancel  buttons.  The  requested  time  limits  and  the  autoDRM  addresses  can 
be  changed  in  the  Confirm  Re-request  window. 


Confirm  Re-request  Window 

This  window  is  generated  by  a  Re-request  option  from  either  the  reqstat  main 
window  or  the  stations  request  window.  The  time  limits  of  the  re-request,  tbeg 
and  tend,  and  the  autoDRM  address  can  be  changed  before  confirming  the  re¬ 
request.  Enter  edit-mode  with  a  right-mouse-button  click  in  any  text  field.  The 
Address  option  is  a  tool  for  changing  the  autoDRM  address  for  multiple  channels 
in  the  Confirm  Re-request  window.  First  select  channels  in  the  Confirm  Re¬ 
request  window  with  a  left-mouse-button  (or  ctrl-left-mouse-button)  click.  Enter  an 
address  in  the  Enter  New  Address  text  field  of  the  Address  popup  or  select  one 
from  the  list  of  autoDRM  sites.  The  list  of  autoDRM  addresses  is  generated  from  the 
unique  addresses  in  the  $REQDATA_HOME/tables/static/global. address  file. 
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HANDLING  ERRORS 


Error  messages  from  reqdata  and  reqchan  are  written  directly  to  the  screen  (stderr). 
Most  of  the  errors  are  caused  by  missing  arguments,  missing  table  files,  invalid  file 
permissions  or  invalid  file  formats.  The  fatal  errors  from  the  programs  reqdata  and 
reqchan  and  the  exit  codes  generated  are  listed  in  the  following  two  tables. 


Code 

reqdata  Errors 

i 

Missing  argument  retaddr 

2 

Missing  argument  basedir 

3 

Invalid  time  argument 

4 

Missing  stachanList  or  stachanFile  argument 

5 

No  stations  or  channels  found  in  stachanFile 

6 

No  stations  or  channels  found  in  stachanList 

7 

Invalid  begPhase  argument 

8 

Invalid  endPhase  argument 

9 

No  iaspeiTable  specified 

10 

No  siteTable  specified 

11 

No  sitechanTable  specified 

12 

No  aff iliationTable  specified 

13 

No  addressTable  specified 

14 

No  lastidTable  specified 

15 

No  requestTable  specified 

16 

Cannot  open  iaspeiTable .hed 

17 

Cannot  open  iaspeiTable .tbl 

18 

Cannot  open  tmpfile 

19 

CSS3.0  origin  free-format  error  in  origin  input  string 

20 

Origin  time  value  missing  or  null 

21 

Origin  lat  value  missing  or  null 

22 

Origin  Ion  value  missing  or  null 

23 

Origin  depth  value  missing  or  null 

24 

Cannot  open  origin  input  file 

25 

Malloc  error. 

26 

CSS3.0  origin  format  error  in  origin  input  file 

27 

Cannot  stat  origin  input  file 

28 

No  origins  specified 

29 

Both  stachanList  and  stachanFile  specified 

30 

Cannot  open  stachanFile 
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Code 

reqdata  Errors  Continued 

31 

Cannot  stat  af f iliationTable 

32 

Cannot  open  aff iliationTable 

33 

CSS3.0  Format  error  in  aff iliationTable 

34 

Cannot  stat  sitechanTable 

35 

Cannot  open  sitechanTable 

36 

CSS3.0  format  error  in  sitechanTable 

37 

Cannot  open  addressTable 

38 

Cannot  stat  siteTable 

39 

Cannot  open  siteTable 

40 

CSS3.0  format  error  in  siteTable 

41 

Requested  station  not  found  in  siteTable 

42 

No  autoDRM  address  for  requested  sta/chan 

43 

Unknown  format  field  in  addressTable 

44 

Cannot  open  requestTable 

45 

No  travel  time  for  begPhase  at  requested  station 

46 

No  travel  time  for  endPhase  at  requested  station 

47 

Cannot  open  pref  ix.reqstatus  file 

48 

Cannot  open  prefix.reqlog  file 

49 

Error  computing  travel  time 

50 

Error  getting  next id 

51 

Cannot  execute  sendmail  program 

52 

Sendmail  program  failed 

Code 

reqchan  Errors 

1 

Missing  argument  retaddr 

2 

Missing  argument  address 

3 

No  lastidTable  specified 

4 

No  requestTable  specified 

5 

Cannot  open  requestTable 

6 

Cannot  open  log  file 

7 

Cannot  get  next id 

8 

Cannot  execute  sendmail  program 

9 

Sendmail  program  failed 

When  an  error  occurs  during  the  processing  of  an  autoDRM  email  response,  the 
error  is  logged  and  the  email  response  is  saved.  The  message  can  be  saved  in  a  file  or 
forwarded  to  the  user’s  mailbox.  To  instruct  the  deliver  program  to  forward  all  email 
that  fails  to  be  processed  to  the  user’s  mailbox,  set  the  variable  FORWARD  -FAILED  _MSG 
to  ‘yes’  in  the  .deliver  script. 
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FORWARD -FAILED _MSG=yes 


By  default,  autoDRM  email  that  is  successfully  processed  is  discarded.  To  save 
all  autoDRM  email  messages,  set  the  variable  SAVE_ALL_GSE_MESSAGES  to  ‘yes’  in  the 
.deliver  script. 

SAVE_ALL_GSE_MESSAGES=yes 

Each  autoDRM  message  will  then  be  saved  in  a  separate  file  in  the  directory 
$REQDATA_HOME/ save. 

An  error  in  the  .deliver  script  which  prevents  the  processing  of  all  messages, 
causes  a  description  of  the  error  to  be  mailed  to  the  user.  Other  less  severe  er¬ 
rors  encountered  by  the  .deliver  script  are  logged  in  the  file  $REQDATA_HOME/logs/ 
deliver .  log.  Error  messages  from  the  message  parsing  program  gse2css  are  logged 
in  the  prefix. reqlog  file  in  the  basedir  directory.  These  can  be  reviewed  with 
the  program  reqstat.  Error  messages  from  the  autoDRM  are  also  logged  in  the 
prefix. reqlog  file.  The  fatal  errors  from  the  programs  gse2css  and  gse2site  and 
the  exit  codes  generated  are  listed  in  the  following  two  tables. 
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Code 

gse2css  Errors 

1 

Cannot  open  reqLog 

2 

Missing  argument  basedir 

Missing  argument  prefix 

Missing  argument  dir 

5 

Length  of  argument  dir  >  64 

6 

Missing  argument  lastidTable 

7 

Cannot  open  input  file 

8 

Cannot  open  reqstatus  file 

9 

Cannot  open  tmpfile 

CSS3.0  origin  format  error 

Get  nextid  failed 

12 

Cannot  open  .origin  file 

13 

Write  to  .origin  file  failed 

14 

Cannot  open  .wfdisc  file 

■a 

GSE  WID1  format  error 

■EH 

GSE  WID2  format  error 

17 

No  GSE  WID  header  found 

18 

Malloc  error 

19 

GSE  CHK2  line  not  found 

20 

Unknown  compression  format 

21 

Cannot  open  dfile  (.w  file) 

22 

Write  to  .wfdisc  file  failed 

23 

Write  to  dfile  failed 
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Code 

gse2site  Errors 

1 

Missing  argument  address 

2 

Missing  argument  lastidTable 

3 

Missing  argument  siteTable 

4 

Missing  argument  sitechanTable 

5 

Missing  argument  addressTable 

6 

Cannot  open  input  file 

7 

Cannot  open  logFile 

8 

Cannot  open  siteTable 

9 

Malloc  error 

10 

CSS3.0  format  error  in  siteTable 

11 

Cannot  open  sitechanTable 

12 

CSS3.0  format  error  in  sitechanTable 

13 

Cannot  open  addressTable 

14 

Format  error  in  addressTable 

15-19 

Error  parsing  DATA_TYPE  STATION 

20 

Write  to  siteTable  failed 

21-25 

Error  parsing  DATA.TYPE  CHANNEL 

26 

Write  to  sitechanTable  failed 

27 

Write  to  addressTable  failed 
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